Metric Entropy and Minimax Risk in Classi cation
نویسندگان
چکیده
We apply recent results on the minimax risk in density esti mation to the related problem of pattern classi cation The notion of loss we seek to minimize is an information theoretic measure of how well we can predict the classi cation of future examples given the classi cation of previously seen examples We give an asymptotic characterization of the minimax risk in terms of the metric entropy properties of the class of distributions that might be generating the examples We then use these results to characterize the minimax risk in the special case of noisy two valued classi cation problems in terms of the Assouad density and the Vapnik Chervonenkis dimension
منابع مشابه
Minimax Nonparametric Classi cation|Part I: Rates of Convergence
|This paper studies minimax aspects of nonparametric classi cation. We rst study minimax estimation of the conditional probability of a class label, given the feature variable. This function, say f; is assumed to be in a general nonparametric class. We show the minimax rate of convergence under square L2 loss is determined by the massiveness of the class as measured by metric entropy. The secon...
متن کاملRisk bounds for Statistical Learning
We propose a general theorem providing upper bounds for the risk of an empirical risk minimizer (ERM).We essentially focus on the binary classi cation framework. We extend Tsybakovs analysis of the risk of an ERM under margin type conditions by using concentration inequalities for conveniently weighted empirical processes. This allows us to deal with other ways of measuring the sizeof a clas...
متن کاملMetric Entropy and Minimax Risk in Classification
We apply recent results on the minimax risk in density estimation to the related problem of pattern classiication. The notion of loss we seek to minimize is an information theoretic measure of how well we can predict the classiication of future examples, given the classiication of previously seen examples. We give an asymptotic characterization of the minimax risk in terms of the metric entropy...
متن کاملA general minimax result for relative entropy
Suppose Nature picks a probability measure P on a complete separable metric space X at random from a measurable set P fP g Then without knowing a statistician picks a measure Q on X Finally the statistician su ers a loss D P jjQ the relative entropy between P and Q We show that the minimax and maximin values of this game are always equal and there is always a minimax strategy in the closure of ...
متن کاملEvaluating techniques for generating metric-based classification trees
Metric-based classi cation trees provide an approach for identifying user-speci ed classes of high-risk software components throughout the software lifecycle. Based on measurable attributes of software components and processes, this empirically guided approach derives models of problematic software components. These models, which are represented as classi cation trees, are used on future system...
متن کامل